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PREFACE 


This  document  is  the  evaluation  plan  for  assessing  the  cost/ 
training  effectiveness  of  the  6883  simulator,  as  compared  to  the  actual 
6883  test  station  equipment  for  training  intermediate  level  F-lll 
avionics  maintenance  personnel  at  Lowry  AFB.  The  project  is  being  con¬ 
ducted  for  the  Air  Force  Human  Resources  Laboratory,  Air  Force  Systems 
Command,  United  States  Air  Force,  Brooks  Air  Force  Base,  Texas.  The 
evaluation  plan  outlined  in  this  report  was  developed  by  the  Social 
Systems  Research  and  Evaluation  Division  of  the  Denver  Research  Insti¬ 
tute,  University  of  Denver,  Denver,  Colorado,  under  Contract  Number 
F33615-78-C-0018.  Dr.  Louis  F.  Cicchinelli  is  the  Principal  Investi¬ 
gator  and  overall  Project  Director. 

This  work  is  being  conducted  under  Project  Number  2361-02-01. 

Dr.  J.  Deignan  is  the  Task  Scientist.  Major  D.  Downing  of  the  Human 
Resources  Laboratory,  Lowry  Air  Force  Base  is  the  Contract  Monitor. 

The  author  wishes  to  acknowledge  the  assistance  of  the  indi¬ 
viduals  who  contributed  to  this  effort.  Mr.  K.  Harmon  and  Mr.  C.  Treese 
of  the  Denver  Research  Institute  have  expended  a  great  deal  of  effort 
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SUMMARY 


Objective 

The  objective  of  this  study  is  to  design  and  implement  a  compre¬ 
hensive  cost/training  effectiveness  evaluation  of  the  6883  simulator, 
as  compared  to  the  use  of  operational  6883  test  station  equipment  for 
training  intermediate  level  (I-level)  F-lll  avionics  maintenance  per¬ 
sonnel.  This  report  concludes  Phase  I  of  the  project  and  provides  the 
detailed  evaluation  plan  to  be  implemented  during  Phase  II. 

f\ 

Approach 


The  evaluation  plan  was  developed  in  view  of  the  constraints 
imposed  by  both  the  training  and  field  environments.  To  ensure  that 
the  evaluation  plan  could  be  implemented  without  major  disruption  to 
existing  training  procedures,  it  was  necessary  to  review  the  evolution 
of  the  F-lll  maintenance  training  course  since  the  specifications  for 
the  simulator  were  developed  two  years  ago.  To  determine  if  the  simu¬ 
lator,  as  delivered  to  Lowry  Air  Force  Base  in  June  1978,  could  be  used 
to  meet  current  training  objectives,  it  is  necessary  to  assess  the 
extent  to  which  various  simulator  capabilities  would  be  operational  at 
the  beginning  of  Phase  II  of  the  project.  This  information  was  col¬ 
lected  by  reviewing  all  relevant  documentation,  interviews  with  Air 
Training  Command  personnel,  and  direct  observation  of  class  proceedings 

Since  the  evaluation  is  also  expected  to  assess  the  relative 
impact  of  the  training  mode  (simulator  vs.  actual  test  station  equip¬ 
ment)  on  subsequent  performance  in  the  field,  it  was  necessary  to 
conduct  site  visits  to  Cannon  and  Plattsburgh  Air  Force  Bases.  The 
primary  purpose  of  the  site  visits  was  to  determine  the  likelihood 
that  job  performance  could  be  objectively  measured,  and  attributed  to 
the  mode  of  training  used  in  relation  to  the  6883  test  station. 

The  final  aspect  of  the  evaluation  will  address  the  relative 
costs  associated  with  using  the  simulator  as  compared  to  the  actual 
test  station  for  training.  The  cost  model  will  be  comprehensive  so 
that  it  can  be  used  to  determine  the  relative  costs  of  the  two  systems 
in  a  variety  of  decision  making  contexts. 

Specif ics 

An  evaluation  plan  was  designed  to  address  the  following 
eight  hypotheses: 


•  Practical  training  on  the  6883  simulator  and  the 
6883  test  station  results  in  identical  performance 
on  the  standard  ATC  block  tests  for  subsequent 
training ; 

•  Airmen  trained  on  the  6883  simulator  and  ti.e  6883 
test  station  are  equally  accurate  in  solving 
trouble-shooting  problems; 

•  Airmen  trained  on  the  6883  simulator  and  the  6883 
test  station  are  equally  efficient  in  solving 
trouble-shooting  problems; 

•  Airmen  trained  on  the  6883  simulator  and  the  6883 
test  station  operate  the  actual  test  station  with 
equal  proficiency; 

•  Airmen  trained  on  the  6883  simulator  and  the  6883 
test  station  are  equally  familiar  and  comfortable 
in  operating  the  actual  test  station  without 
supervision; 

•  Airmen  trained  on  the  6883  simulator  and  the  6883 
test  station  will  acquire  equivalent  job-related 
experience; 

•  Airmen  trained  on  the  6883  simulator  and  the  6883 
test  station  will  be  equally  capable  of  operating 
the  6883  test  station  in  the  field;  and 

•  Airmen  trained  on  the  6883  simulator  and  the  6883 
test  station  are  equally  capable  of  operating 
assigned  test  stations,  other  than  the  6883 
station,  in  the  field. 

In  this  evaluation  design,  the  mode  of  training  is  the  experi¬ 
mental  treatment.  Specifically,  the  use  of  the  6883  simulator  and  the 
6883  test  station  in  the  practical  block  of  instruction  in  course 
3ABR32634A  are  the  two  treatments.  A  comparison  of  classroom  and  field 
performance  for  the  two  treatment  groups  is  expected  to  answer  the 
general  question:  Do  airmen  trained  on  the  6883  simulator  perform 
differently  than  airmen  trained  on  the  actual  test  station  equipment? 
Due  to  a  variety  of  environmental  constraints,  it  will  be  necessary  to 
consider  16  different  groups  in  conducting  some  of  the  proposed  analy¬ 
ses.  These  groups  are  defined  by  assigning  students  one  of  two  levels 
of  four  variables  of  concern:  training  sequence,  training  mode,  test¬ 
ing  mode,  and  field  assignment. 

A  total  population  of  180  F-lll  avionics  maintenance  trainees 
is  anticipated  during  the  one-year  evaluation  period.  Students  will 
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be  assigned  to  the  eight  basic  treatment  groups  defined  by  the  levels 
of  training  sequence,  training  mode,  and  test  mode.  To  ensure  random 
assignment  to  the  treatment  group.',  a  random  assignment  plan  will  be 
implemented.  The  final  factor  to  be  considered  which  relates  to  the 
assignment  of  students  to  treatment  groups  is  the  procedure  used  to 
make  field  assignments.  Evaluator  control  over  this  factor  is  not 
possible  since  assignments  are  made  on  the  basis  of  demand  in  the  field. 
Thus,  any  data  collected  subsequent  to  field  assignments  will  be  ana¬ 
lyzed  for  airmen  assigned  to  the  6883  or  6873  test  station  and  airmen 
assigned  to  other  automatic  test  stations  separately. 

Approximately  eight  additional  hours  of  training  will  be  re¬ 
quired  to  collect  performance  data  on  each  class  of  students.  This 
test  session  will  be  added  to  the  6883  practical  block  of  instruction 
resulting  in  a  total  of  five  days  for  that  block  of  instruction. 

Conclusions 

The  evaluation  plan  presented  in  this  report  can  be  implemented 
in  the  current  training  environment  at  Lowry  AFB.  The  results  of  the 
evaluation  will  highlight  the  relative  advantages  and  disadvantages  of 
the  6883  simulator  as  compared  to  the  use  of  actual  test  station 
equipment  in  training.  It  will  also  be  possible  to  isolate  any 
differences  in  maintenance  personnel  performance  in  the  field  which 
are  due  to  the  nature  of  training  received,  if  any  differences  exist. 
Finally,  the  cost  model  developed  will  be  exercised  in  a  number  of 
hypothetical  decision  making  contexts  to  illustrate  its  flexibility 
and  generalizab ility .  The  model  itself  will  serve  as  a  methodology  for 
making  comparative  cost  assessments  in  making  future  choices  between 
the  use  of  simulation  and  actual  equipment  in  training. 


INTRODUCTION 


The  following  evaluation  plan  has  been  developed  in  view  of 
information  collected  by  the  Denver  Research  Institute  (DRI)  evaluation 
team  over  the  past  six  months.  While  the  general  framework  remains 
consistent  with  the  proposed  contract  statement  of  work,  some  minor 
modifications  have  been  made  in  the  manner  of  implementation  based  on 
an  analysis  of  the  training  and  field  environments.  The  overall  plan 
is  divided  into  three  components  to  facilitate  reference  to  the  three 
major  evaluation  components  originally  outlined  in  the  proposal. 
Specifically,  the  three  major  components  of  the  evaluation  are: 

•  Classroom  performance; 

•  Field  performance;  and 

•  Cost  analysis. 

This  report  is  divided  into  five  sections.  The  first  section 
outlines  the  overall  design  to  be  employed,  the  next  three  sections 
correspond  to  the  three  major  components  of  the  evaluation  plan,  and 
a  fifth  section  provides  a  chart  summarizing  the  hypotheses  to  be 
tested  and  the  data  collection  instruments  and  performance  measures 
to  be  used.  In  each  of  the  component-related  sections,  the  primary 
objective  and  specific  hypotheses  to  be  tested  are  stated.  Next,  the 
findings  of  the  preliminary  investigations  which  most  directly  relate 
to  that  evaluation  component  are  discussed.  Finally,  in  view  of  those 
findings  and  the  proposed  evaluation  effort,  the  methodology  to  be 
employed  is  outlined. 
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EVALUATION  FRAMEWORK 


From  a  theoretical  point  of  view,  the  evaluation  design 
required  to  assess  classroom  and  field  performance  differences  between 
simulator  and  actual  equipment-trained  students  is  extremely  complex. 
The  complexity  stems  primarily  from  two  factors:  the  natural  setting 
in  which  the  evaluation  will  occur  and  the  difficulty  in  defining  the 
treatments.  These  issues  will  be  discussed  in  detail  throughout  this 
section. 


At  a  general  level,  the  design  to  be  employed  approximates  a 
"randomized  controlled  field  trial"  approach  to  evaluation.  In  a 
discussion  of  this  methodology,  Gilbert,  et  al.  (1975)  point  out 
that  the  experimental  units  (students)  "are  randomly  assigned  to 
treatments  or  regimens  and  carefully  followed  to  find  out  what  the 
effect  of  the  regimen  might  be."  The  "randomized  controlled  field 
trial"  model  includes  three  basic  assumptions.  First,  the  term 
"field  trial"  implies  that  treatments  are  being  implemented  in  the 
field  rather  than  simulated  in  the  laboratory.  Second,  the  term 
"controlled"  implies  that  either  the  investigator  or  natural  processes 
dictate  the  nature  of  treatments  to  be  administered.  Third,  "ran¬ 
domized"  refers  to  the  use  of  chance  to  assign  experimental  units  to 
specific  treatment  groups. 


Treatment 


Strictly  speaking,  in  the  present  case,  the  mode  of  training  is 
the  experimental  treatment.  Specifically,  the  use  of  the  6883  simula¬ 
tor  and  the  6883  test  station  in  the  practical  block  of  instruction 
in  course  3ABR32634A  are  the  two  treatments.  A  comparison  of  classroom 
and  field  performance  for  the  two  treatment  groups  is  expected  to 
answer  the  question,  "Do  airmen  trained  on  the  6883  simulator  perform 
differently  than  airmen  trained  on  the  actual  test  station  equipment?" 

It  should  be  obvious  that  the  anticipated  impacts  of  training 
are  many.  For  example,  students  are  expected  to  gain  an  understanding 
of  electronics  in  general  and  to  learn  the  operation  and  maintenance  of 
the  test  station  in  both  theory  and  practice.  Further,  some  impacts 
will  be  immediate  and  can  be  measured  at  the  completion  of  training; 
others  will  be  more  long  range  and  can  be  measured  only  after  field 
assignments  are  made.  In  short,  to  assess  the  impact  of  training,  a 
variety  of  performance  measures  are  required  at  a  number  of  logical 
points  in  time. 

The  measurement  of  performance  over  time  often  presents  a 
problem  in  designing  evaluation  studies.  Campbell  and  Stanley  (1963) 
point  out  that  the  measurement  of  performance  itself  can  affect 
future  performance.  This  is  particularly  true  when  assessing  the 
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impact  of  training  since  testing  can  be  considered  additional  training. 
To  further  complicate  the  problem,  the  6883  simulator  provides  training 
and  testing  capabilities  not  available  on  the  actual  test  station 
equipment.  Thus,  it  is  necessary  to  use  the  simulator  for  performance 
testing,  in  particular  for  measuring  trouble-shooting  performance.  At 
the  same  time,  it  is  important  to  realize  that  students  will  be 
assigned  to  actual  test  stations  in  the  field.  Therefore,  it  is 
necessary  to  measure  performance  using  actual  equipment  to  ensure  that 
performance  levels  are  not  due  to  the  idiosyncracies  of  the  simulated 
test  station  and  its  environment.  In  sum,  to  assess  performance  in 
the  classroom,  the  two-by-two  experimental  design  shown  in  Figure  1 
is  required.  Treatment  at  this  point  in  time  is  the  mode  of  training 
in  the  6883  practical  block  of  instruction. 


Training 


Simulator 


Testing 


Actual 

Equipment 


Simulator  Actual  Equipment 


A 

B 

C 

D 

Figure  1.  Basic  Evaluation  Design 


Another  aspect  of  the  evaluation  is  to  determine  if  any  differ¬ 
ences  exist  in  performance  in  the  field  as  a  function  of  treatment.  In 
measuring  field  performance  at  this  point  in  time,  it  is  important  to 
note  that  four  treatments  must  be  considered.  That  is,  the  combina¬ 
tions  of  training  and  performance  testing  shown  in  Figure  1  constitute 
four  distinct  treatments.  One  additional  factor  further  complicates 
the  definition  of  treatment  when  assessing  field  performance:  the 
nature  of  the  field  assignment.  For  our  purposes,  two  types  of 
assignments  will  be  considered:  (a)  assignment  to  either  the  6883 
or  68731  test  station  or  (b)  assignment  to  other  automatic  F-lll 


The  6883  and  6873  test  stations  are  identical  except  that  the 
6873  station  includes  an  astrotracking  test  panel.  This  equipment  is 
available  only  on  the  F-lll  FB  model  aircraft.  Training  in  the  opera¬ 
tion  and  maintenance  of  this  station  is  identical  since  the  operation 
of  the  astrotracking  test  equipment  is  learned  on  the  job. 
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test  stations.  This  dichotomy  must  be  made  because  it  is  reasonable 
to  assume  that  tne  impact  of  the  mode  of  6883  test  station  training 
would  be  more  dramatic  for  personnel  assigned  to  identical  test  sta¬ 
tions  than  for  personnel  assigned  to  similar,  but  different,  automatic 
test  stations.  In  sum,  any  assessment  of  performance  in  the  field  must 
be  made  in  view  of  the  eight  treatment  groups  shown  in  Figure  2. 


Trolnlng 


Figure  2.  Treatment  Groups  for  Assessment 
of  Field  Performance 


The  last  factor  which  must  be  considered  when  assessing  the 
impact  of  simulator  training  is  the  sequence  of  training  immediately 
preceding  testing.  Recall  that  the  practical  6883  training  is  only 
one  portion  of  the  total  F-lll  avionics  maintenance  course.  In  fact, 
the  treatment  (simulator  vs.  actual  equipment)  will  only  be  adminis¬ 
tered  for  three  consecutive  days.  Since  this  constitutes  such  a  small 
portion  of  the  training  experience,  it  will  be  necessary  to  develop  a 
design  which  will  be  sensitive  to  even  slight  differences  in  perfor¬ 
mance.  Of  particular  concern  then  is  the  time  between  basic  treatment 
(training)  and  testing.  As  the  course  is  currently  structured,  all 
blocks  of  instruction  wil^be  administered  in  a  consistent  sequence 
for  both  treatment  groups.  The  major  problem  is  within  the  6883 


Unscheduled  deviations  in  the  training  sequence  often  occur, 
usually  due  to  equipment  malfunctions.  Since  there  can  be  no  control 
over  this  factor  and  the  occurrence  of  malfunctions  is  random,  the 
impact  on  performance  is  expected  to  be  consistent  across  treatment 
groups. 
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practical  block  itself.  The  duration  of  this  instruction  is  six 
days:  three  days  of  training  on  the  6883  test  station  and  three  days 
on  the  6886  test  station.  To  maximize  the  amount  of  student  contact 
with  both  test  stations,  each  class  will  be  divided  into  two  groups. 

One  group  will  receive  training  on  the  6883  station  for  three  days 
and  then  receive  training  on  the  6886  station  for  three  days. 
Simultaneously,  the  second  group  will  receive  the  same  training  but  in 
reverse  order:  three  days  on  the  6886  station  and  then  three  days  on 
the  6883  station.  Since  all  testing  associated  with  this  evaluation 
will  occur  after  the  sixth  day  of  training,  some  students  will  have 
6886  test  station  training  intervening  between  6883  training  and  6883 
testing.  It  is  reasonable  to  assume,  therefore,  that  recency  effects 
and/or  retroactive  interference  may  confound  performance  measures 
made  at  the  end  of  the  instruction  block.  To  control  for  the  potential 
impact  of  training  sequence  on  performance,  this  variable  will  be 
included  in  the  definition  of  treatment  groups.  This  procedure  is 
equivalent  to  employing  a  counterbalanced  design  to  randomize  the 
effect  of  training  sequence.  Additionally,  the  procedure  will  allow 
examination  of  the  extent  of  impact. 

In  total  then,  two  levels  of  four  variables  will  be  used  to 
define  16  independent  experimental  groups  (Figure  3) . 


Assignment  to  Treatment  Groups 

A  total  population  of  180  F-lll  avionics  maintenance  trainees 
is  anticipated  during  the  one-year  evaluation  period.  Students  will 
be  assigned  to  the  eight  treatment  groups  defined  by  the  levels  of 
training  sequence,  training  mode,  and  test  mode  as  shown  in  Figure  3. 
To  ensure  an  approximately  equal  number  of  subjects  in  each  treatment 
group,  the  assignment  plan  shown  in  Table  1  will  be  implemented.  From 
the  table^  it  can  be  seen  that  "training  class"  is  the  unit  of  assign¬ 
ment  to  a  level  of  training  sequence  and  training  mode;  whereas 
"student"  is  the  unit  of  assignment  to  a  level  of  test  mode.  The  plan 
shown  in  Table  1  is  for  a  one-month  period  and  assignment  of  units 
to  treatments  is  essentially  random.  It  is  expected  that  the  schedule 
will  be  continually  adjusted  in  subsequent  months  to  maintain  approxi¬ 
mately  equal  group  sizes  throughout  the  data  collection  period.  Since 
the  adjustment  will  be  most  likely  necessitated  by  randomly  occurring 
equipment  malfunctions  and  changes  in  training  schedules,  the  overall 
attempt  to  randomly  assign  students  to  treatment  groups  will  not  be 
compromised.  Table  2  is  the  student  assignment  sheet  which  will  be 
submitted  to  ATC  personnel  for  each  class. 

Despite  the  effort  to  randomly  assign  students  to  groups, 
it  is  possible  that  observed  performance  will  be  a  function  of  student 
aptitude.  To  justify  attributing  observed  differences  (if  any)  to 
training  mode,  it  is  necessary  to  consider  preexisting  individual 
differences  among  students.  To  address  this  issue,  an  analysis  of  the 
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Four  Variables  Which  Define  the  Treatment  Regimen 


TABLE  2 


STUDENT  ASSIGNMENT  TO  TRAINING 


Class  Number: 


Number  of  Students: 


6886  practical  then  6883  practical  on  actual  equipment 


6883  practical  instructor: _ 


FOR  DRI  USE  ONLY 


Group  Number 


6886  practical  then  6883  practical  on  simulator 


6883  practical  instructor: _ 


6883  practical  on  simulator  then  6886 


6883  practical  instructor: _ 


6883  practical  on  the  actual  test  station  then  6886 


relationship  between  aptitude  scores  and  scores  on  block  exams  and 
performance  in  the  6883  theory  and  practical  blocks  of  instruction  was 
recently  completed  on  104  airmen  trained  over  the  past  two  years. 

The  analysis  indicates  that  only  modest  correlations  exist  between 
aptitude  as  measured  by  Air  Force  Testing  protocols  and  performance  in 
the  6883  training  blocks.  It  is  important  to  note  that  this  analysis 
was  based  on  previous  block  examinations  which  have  been  continually 
revised  to  reflect  changes  in  course  content.  Therefore,  these  findings 
are  not  necessarily  generallzable  to  the  test  instruments  to  be  used  in 
this  evaluation  effort.  Thus,  although  no  systematic  bias  due  to 
aptitude  is  expected  to  confound  the  performance  measures,  these  differ¬ 
ences  will  be  controlled  by  including  an  analysis  of  covariance  in  the 
proposed  data  analysis  plan. 

The  final  issue  to  be  considered  which  relates  to  the  assign¬ 
ment  of  students  to  treatment  groups  is  the  procedure  used  to  make 
field  assignments  ( cf .  pp.  3-4).  Evaluator  control  over  this  factor 
is  not  possible  since  assignments  are  made  on  the  basis  of  demand  in 
the  field.  Thus,  any  data  collected  subsequent  to  field  assignments 
will  be  analyzed  separately  for  airmen  assigned  to  the  6883  or  6873 
test  station  and  airmen  assigned  to  other  automatic  test  stations.  It 
is  expected  that  the  group  sizes  will  be  unequal  on  this  factor  because 
only  a  small  number  of  trainees  will  be  assigned  to  the  6883  or  6873 
test  stations. 


Testing  Schedule 

Approximately  eight  additional  hours  of  training  will  be 
required  to  collect  proposed  performance  data  on  each  class  of  students. 
This  test  session  will  be  added  to  the  6883/6886  practical  block  of 
instruction,  resulting  in  a  total  of  seven  days  for  this  block  of  in¬ 
struction.  The  sequence  of  events  occurring  on  the  test  day  are  as 
follows : 


•  Trouble-shooting  test  on  either  the  simulated  or  actual 
test  station  (practical);^1 


The  details  of  this  analysis  were  included  as  an  appendix  to 
the  "Second  Quarterly  Status  Report,"  to  Human  Resources  Laboratory, 
Lowry  AFB  (1979) . 

A 

From  Table  1  it  can  be  seen  that  students  in  a  single  class 
are  divided  for  the  purpose  of  testing  trouble-shooting  ability.  This 
procedure  should  result  in  an  overall  reduction  in  the  time  required 
for  testing  because  testing  on  the  simulator  and  actual  test  equipment 
can  be  conducted  concurrently. 
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•  Estimated  Job  Proficiency  Test  administered  to  all 
students;  and 

•  Interviews  with  all  students. 

The  nature  of  these  data  collection  instruments  is  discussed  in 
the  remaining  sections  of  the  report.  In  addition  to  these  performance 
measures,  scores  on  the  routinely  administered  performance  checklists, 
prior  and  subsequent  block  exams,  and  aptitude  tests  will  be 
analyzed . 

It  is  important  to  note  that  while  the  testing  schedule  and 
plan  of  group  assignments  are  specified  at  the  outset,  enough  flexi¬ 
bility  must  be  maintained  so  that  student  reassignment  can  be  made  on 
short  notice  to  reduce  the  occurrence  of  lost  or  missing  data.  To 
assist  in  this  regard,  some  general  guidelines  for  dealing  with  the 
more  typical  problems  will  be  developed.  Additionally,  a  DRI  staff 
member  will  be  made  continually  available  to  assist  in  making  these 
immediate  decisions. 


Data  Analysis 

Initially,  the  performance  data  collection  instruments  will  be 
administered  solely  by  Denver  Research  Institute  (DRI)  personnel,  with 
the  assistance  of  Human  Resources  Laboratory  (HRL)  and  Air  Training 
Command  (ATC)  staff.  Interviews  with  students  and  instructors  will  be 
conducted  by  DRI  staff.  However,  it  is  expected  that  this  information 
will  eventually  be  collected  through  the  use  of  a  self-administered 
questionnaire.  Subjective  ratings  of  field  performance  will  be 
obtained  for  individual  students  from  field  supervisors  using  a  self- 
administered  questionnaire.  Training  in  the  use  of  this  instrument 
will  be  provided  by  DRI. 

All  data  will  be  aggregated  and  analyzed  by  DRI  staff.  Table  3 
is  the  data  collection/coding  form  to  be  used.  This  form  provides  an 
outline  of  all  data  to  be  collected  on  each  student.  The  specific 
analysis  and  measures  to  be  used  are  discussed  throughout  the  remainder 
of  this  report,  as  they  relate  to  specific  hypotheses. 
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Previous  Block  Scores  Aptitude  Scores  Identifying  Information 


TABLE  3 


1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 

11. 

12. 

13. 

14. 

15. 

16. 

17. 

18. 

19. 

20. 

21. 

22. 

23. 

24. 


STUDENT  DATA  COLLECTION  FORM 


Group  Id.  Number 
Student  Name 
Sex 

Date  of  Birth 

Social  Security  Number 

Date  of  Arrival 

Date  of  Entry 

Class  Number 

Course  Sequence  Number 

Instructor  Code 

General 

Mechanical 

Administrative 

Electronics 

AFQT 

Fundamentals  of  Electronics 
Intro,  to  Avionics  AGE  Principles 
Cenpac 
Datac 

Data  Logic  Analysis  of  Counter 
Timer,  Logic  and  Micrologic  Power 
Supplies 

CATE 

Navigation  and  Weapons  Delivery  TS 
Electronic  Systems  TS 
Converter  Flight  Control  TS 


Columns 


1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 

11. 

12. 


13. 


14. 

15. 

16. 

17. 

18. 
19. 


20. 

21. 

22. 

23. 

24. 


1 

2-7 


□ 


□ 

□ 


□CD 


cm 

cm 

cm 

CED 


cm 


8 

9-14 

15-23 

24-29 

30-35 

36-41 

42 

43 

44-46 

47-49 

50-52 

53-55 

56-58 

59-61 

62-64 

65-67 

68-70 

71-73 

74-76 

77-79 

80-82 

83-85 
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Experimental  Data  Special  Information  Subsequent  Block  Scores 


TABLE  3  (cont.) 


Columns 


25. 

26. 

27. 

28. 

29. 

30. 

31. 

32. 

33. 

34. 

35. 

36. 

37. 

38. 

39. 

40. 

41. 

42. 

43. 

44. 

45. 

46. 

47. 

48. 


Electronic  Systems  TS  (pract) 
Converter  Flight  Control  TS  (pract) 
Computer  TS 
Attitude  and  Rate  TS 
Computer  TS  (pract) 

Attitude  and  Rate  TS  (pract) 
Displays  TS 
Video  TS 

Radar-Transmitter-Modulator  TS 
Video  TS  (pract) 

RTM  TS  (pract) 

Date  of  Graduation 
Date  of  Elimination 
Base  Assignment 
Report  Date 
Deviations  in  Training 
Unique  Errors  Committed 


Estimated  Job  Proficiency  Test 
Score 


25.  cm 
26. 1  1  | 
»■!'-]  1. 


33.  |  |  | 

34-1  .1.  1  .1 

35.  cm 


37.1  I  I  I  I  11 
38.  □ 


AO-1  Cl 

41.  CD 

42.  CD 

43. cn 


cm 

cm 

cm 

cm 


44.  |  U 

45. cn 

46.  CO 


47.m 


48.CCO 


86-88 

89-91 

92-94 

95-97 

98-100 

101-103 

104-106 

107-109 

110-112 

113-115 

116-118 

119-124 

125-130 

131 

132-137 

138-139 

140-141 

142-143 

144-145 

146-147 

148-149 

150-151 

152-153 

154-156 
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Experimental  Troubleshooting  Test  Data 


TABLE  3  (cont.) 


49. 

50. 

51. 

52. 

53. 

54. 

55. 

56. 

57. 

58. 

59. 

60. 

61. 

62. 

63. 

64. 

65. 

66. 

67. 


Columns 


Remove  all  jewelry 

49.  □ 

157 

Test  station  preparation 

O 

.□ 

158 

Turn-on  procedures 

51-  □ 

159 

Feel/Trim  visual  inspection 

52.  0 

160 

Selection/hook-up  of  correct 
adapter 

53.0 

161 

Cabling  LRU  to  test  station 

54.  0 

162 

Set  MODE  SWITCH  to  "normal" 

55.  □ 

163 

Enter  and  request  test  number 

830401 

56.  0 

164 

Verify  test  number,  press  START 

57.  □ 

165 

Verify  test  date,  press  START 

58.  0 

166 

Does  LRU  pass  test  #300150 

59.  0 

167 

Testing  halts  at  #300610;  enter 
#300650,  TEST  REQUEST,  START 

60.  O 

168 

Testing  halts  at  #300755;  when  "go" 
press  START,  observe  increasing  + 
voltage,  press  START 

61.  0 

169 

Testing  halts  at  #300764;  when  "go" 
press  START,  observe  increasing  - 
voltage,  press  START 

62.  0 

170 

Testing  halts  at  #300824;  enter 
#300830,  TEST  REQUEST,  START 

63.  O 

171 

Testing  halts  at  #301752;  rotate 
OSCP.  INPUT  SIGNAL  SEL  switch  to 
"digital  multr" 

64.  0 

172 

Press  NEG  DC  COUPLING  switch 

65.  0 

173 

Press  POS  DC  COUPLING  switch 

66.  O 

174 

Set  VERNIER  to  "fully  cw" 

67.  0 

175 
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Experimental  Troubleshooting  Test  Data  (cont. 


TABI.E  3  (cont.) 


Columns 

68. 

Set  V/D1V  at  .5. 

68. 

□ 

176 

69. 

Set  SWEEP  MODE  on  "auto." 

69. 

□ 

177 

70. 

Adjust  to  obtain  display,  observe 

□ 

,  , 

sine  wave 

70. 

178 

71. 

Determine  that  the  Yaw  Board  TB3  is 
defective. 

71. 

□ 

179 

c 

72. 

Rerun  test  tape  from  #301740. 

72. 

□ 

180 

o 

73. 

Advise  board  replacement . 

73. 

□ 

181 

n) 

4J 

«J 

Q 

74. 

Rerun  test  tape  from  #301740. 

74. 

□ 

182 

*J 

m 

H 

75. 

Student  should  offer  alternative 
explanations  of  malfunction. 

75. 

□ 

183 

60 

c 

■H 

76. 

After  5  minutes:  Instructor  suggests 

4J 

O 

O 

JC 

W 

that  "test  station  caused  the 
malfunction. " 

76. 

□ 

184 

0) 
»— 1 
x> 

77. 

After  another  5  minutes:  Instructor 

o 

H 

suggests  that  "the  test  program 
should  be  decoded." 

77. 

□ 

185 

H 

4-* 

c 

78. 

Student  decodes  program  in  total. 

78. 

□ 

186 

1 

79. 

Student  decodes  program  in  part . 

79. 

□ 

187 

<y 

o. 

& 

80. 

Student  identifies  missing  power 
input . 

80. 

□ 

188 

81. 

Student  notes  that  power  light  is 

□ 

.  , 

off. 

81. 

189 

82. 

Student  suggests  alternative 
explanations  for  lack  of  power. 

82. 

□ 

190 

83. 

Student  suggests  that  a  fuse  has 
blown. 

83. 

□ 

191 
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COMPONENT  1 :  SIMULATOR  RELATED  ISSUES 


The  primary  objective  of  this  component  of  the  evaluation  is 
to  determine  the  relative  advantages  and  disadvantages  of  the  6883 
simulator  as  compared  to  the  use  of  actual  test  station  equipment  in 
training.  The  specific  null  hypotheses  to  be  addressed  are: 

Hy  1 — Practical  training  on  the  6883  simulator  and  the 

6883  test  station  results  in  identical  performance 
on  the  standard  ATC  block  tests  for  subsequent 
training . 

Hy  2 — Airmen  trained  on  the  6883  simulator  and  the  6883 
test  station  are  equally  accurate  in  solving 
trouble-shooting  problems. 

Hy  3 — Airmen  trained  on  the  6883  simulator  and  the  6883 
test  station  are  equally  efficient  in  solving 
trouble-shooting  problems. 

Hy  4 — Airmen  trained  on  the  6883  simulator  and  the  6883 
test  station  operate  the  actual  test  station  with 
equal  proficiency. 

Hy  5 — Airmen  trained  on  the  6883  simulator  and  the  6883 
test  station  are  equally  familiar  and  comfortable 
in  operating  the  actual  test  station  without 
supervision . 

In  addition  to  testing  these  specific  research  hypotheses,  the 
students  and  instructors  will  be  surveyed  to  obtain  their  attitudes 
and  perceptions  of  simulation  as  a  training  tool. 


Relevant  Issues 


DRI  originally  proposed  to  rely  heavily  on  existing  test 
instruments  to  collect  relevant  data.  Further,  it  was  proposed  to 
emphasize  training  on  the  6883  simulator  since  previous  test  scores 
from  students  trained  on  the  actual  test  station  equipment  would  be 
made  available  to  serve  as  baseline  data.  However,  after  completing 
a  review  of  courses,  it  was  found  that  the  merger  of  3ARB326XID 
(operations)  and  3ARB326XOB  is  not  the  only  major  change  in  instruc¬ 
tion  over  the  past  year.  In  fact,  these  two  courses  have  undergone 
numerous  modifications  and  the  new  combined  course  is  continually  being 
altered  even  at  this  late  date.  Associated  with  unese  course  changes 
are  changes  in  the  test  instruments  used.  Appendix  A  includes  a  table 
which  shows  the  evolution  of  training  objectives  over  the  past  three 
years  for  the  6883  theory  and  practical  blocks  of  instruction.  Most 
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recently,  the  6883  and  6886  test  station  practical  blocks  have  been 
merged.  These  formal  modifications,  together  with  numerous 
"deviations,"  applied  to  nearly  every  class,  make  the  use  of  test 
scores  for  previous  classes  as  Daseline  data  inappropriate.  The  table 
in  Appendix  B  shows  the  6883  block  objectives  formerly  addressed  and 
included  in  the  X4A  STS  which  are  not  currently  included  in  the  new 
6883  instruction  block.  Additionally,  over  the  past  six  months,  stu¬ 
dents  have  been  sporadically  trained  on  either  the  6883  simulator  or 
the  actual  equipment,  or  both,  depending  on  equipment  availability  and 
convenience.  Thus,  including  data  from  this  group  in  our  analysis  is 
also  unjustified. 

It  should  also  be  noted  that  for  some  instruction  blocks, 
previous  test  scores  do  not  exist.  For  example,  while  the  XOB  (main¬ 
tenance)  course  has  included  an  evaluation  of  student  performance 
during  the  6883  practical  block  by  using  a  performance  criterion  check¬ 
list,  no  similar  testing  procedure  has  been  employed  in  the  X1D  (oper¬ 
ations)  course.  Student  performance  in  this  course  has  been  rated  as 
"satisfactory"  or  "unsatisfactory"  on  the  basis  of  instructor's  obser¬ 
vations  throughout  the  four  days  of  training. 

In  the  new  course  (3ARB32634A) ,  students  will  spend  three  days 
on  the  6883  test  station  during  the  practical  block  of  instruction. 

This  intensified,  but  shortened,  period  of  instruction  will  certainly 
reduce  the  probability  of  observing  major  differences  in  performance 
due  to  training  mode.  Further,  it  will  eliminate  any  "free"  time 
during  which  to  administer  additional  tests  for  evaluation  purposes. 

In  sum,  these  aspects  of  the  training  environment  pose  a  com¬ 
plex  of  potentially  confounding  variables  which  might  obscure  the 
measurements  of  any  differences  in  performance  due  to  the  minimal 
intervention  of  three  days  for  differential  training.  The  following 
methodologies  are  planned  to  overcome  many  of  these  difficulties  and 
to  provide  as  much  temporary  control  over  the  training  environment  as 
possible . 


Methodology 

The  students  in  course  3ARB326X4A  will  be  assigned  to  one  of 
two  groups.  Some  classes  will  receive  the  6883  practical  instruction 
exclusively  on  the  simulator,  while  the  remaining  students  will  be 
trained  exclusively  on  the  actual  test  station  equipment .  The  sequence 
of  training  in  the  6883  practical  block  will  be  controlled  such  that 
approximately  one-half  of  the  students  will  receive  each  of  the  two 
possible  sequences.  Half  of  each  class  will  then  be  tested  on  either 
the  simulator  or  the  actual  equipment.  This  will  result  in  a  2  x  2 
design  for  each  training  sequence  as  shown  in  Figure  1. 
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Data  Collection  Instruments 


The  data  collection  effort  associated  with  this  component  of 
the  evaluation  plan  includes  the  use  of  existing  performance  tests  as 
well  as  new  data  collection  instruments. 

As  originally  proposed,  data  will  be  collected  using  existing 
Air  Force  performance  tests  when  appropriate.  Therefore,  the  evalua¬ 
tion  of  some  hypotheses  will  be  based  on  the  use  of  currently  used 
end-of-block  tests.  Specifically,  the  performance  on  the  end-of-block 
test  for  the  6883  practical  block  will  be  used  to  compare  the  perform¬ 
ance  of  the  treatment  groups.  Additionally,  to  examine  the  possibility 
that  the  mode  of  training  affects  subsequent  classroom  performance  in 
non-6883  blocks,  end-of-block  scores  for  all  training  following  the 
6883  practical  block  will  be  compared  foi  the  groups. 

A  Trouble-shooting  Test  will  be  designed  such  that  it  can  be 
administered  on  the  simulator  and  the  actual  test  station  equipment  in 
exactly  the  same  manner.  The  test  has  been  designed  to  allow  examina¬ 
tion  of  a  student's  ability  to  trouble-shoot  Line  Replaceable  Units 
(LRUs)  and  the  test  station  as  an  integrated  system.  The  performance 
measures  to  be  used  are  accuracy,  time  to  completion,  and  nature  of 
errors.  The  test  protocol  will  include  familiar  and  novel  aspects. 

The  data  recording  sheet  associated  with  this  test  is  shown  as  Table  4. 
If  sufficient  data  is  collected  before  the  one-year  collection  period 
is  over,  one  portion  of  the  test  administered  on  the  6883  simulator  will 
be  modified  to  include  a  problem  which  cannot  be  presented  on  the  actual 
test  station.  Specifically,  the  simulated  Yaw  Computer  trouble-shooting 
exercise  will  be  included  to  examine  the  ability  of  students  to  general¬ 
ize  their  training  to  an  LRU  with  which  no  experience  exists. 

Finally,  a  structured  interview  format  will  be  developed  to 
obtain  student  attitudes  and  perceptions  concerning  the  use  of  the  6883 
simulator  versus  the  actual  test  station  in  training.  All  students 
trained  and  tested  on  alternative  equipment  modes  of  presentation  will 
be  interviewed.  To  gain  insight  in  the  use  of  simulation  in  training 
from  the  instructor's  point  of  view,  all  ATC  personnel  who  have  used 
both  modes  of  training  will  be  interviewed. 


Hypothesis  Testing 

To  assess  hypothesis  It  1,  the  scores  of  the  two  treatment  groups 
on  the  6883  practical  block  test  will  be  compared,  assuming  a  separate 
test  is  administered  by  ATC.  Additionally,  the  end-of-block  scores  for 
subsequent  blocks  (non-6883  instruction)  of  instruction  will  be  compared 
in  an  effort  to  isolate  any  secondary  impacts  of  training  mode  of  sub¬ 
sequent  performance.  All  comparisons  of  subsequent  performance  will  be 
made  among  the  four  treatment  groups  shown  in  Figure  1.  It  is  expected 
that  an  analysis-of-variance  model  will  be  devised  to  assess  this 
hypothesis . 
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TABLE  4. 


TROUBLE -SHOOT INC  TEST 
DATA  RECORDING  SHEET 


Student  Name _ 

Observer 

ERROR  NOTED  (indicate 
step  #  and  ycur  comment) 


12.  Testing  halts  at  #300610;  enter 
#300650,  TEST  REQUEST,  START. 

13.  Testing  halts  at  #300755;  when  "go," 
press  START,  observe  increasing  + 
voltage,  press  START. 

14.  Testing  halts  at  #300764;  when  "go," 
press  START,  observe  increasing  - 
voltage,  press  START. 

15.  Testing  halts  at  #300824;  enter 
#300830,  TEST  REQUEST,  START. 

Note:  Test  station  only — "jump" 
testing  sequence  to  #301400. 

16.  Testing  halts  at  #301752;  rotate  OSCP, 
INPUT  SIGNAL  SEL  switch  to  "digital 
multr . " 


17.  Press  NEG  DC  COUPLING  switch. 

18.  Press  POS  DC  COUPLING  switch. 

19.  Set  VERNIER  to  "fully  cw." 


Note:  Test  station  only — "jump" 
testing  sequence  to  #300600. 


Note  Time  Steps  Completed  Yes  No 

0  .... 

1.  Remove  all  jewelry.  _  _ 

2.  Test  station  preparation.  _  _ 

3.  Turn-on  procedures.  _  _ 

4.  Feel/Trim  visual  inspection.  _  _ 

5.  Selection/hook-up  of  correct  adapter.  _  _ 

6.  Cabling  LRU  to  test  station.  _  _ 

7.  Set  MODE  SWITCH  to  "normal."  _  _ 

8.  Enter  and  request  test  #830401.  _  _ 

9.  Verify  test  number,  press  START.  _  _ 

10.  Veriry  test  date,  press  START.  _  _ 


11.  Does  LRU  pass  test  #300150. 
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TABLE  A.  (cont.) 


23.  Determine  that  the  Yaw  Board  TB3 
is  defective. 

2A.  Rerun  test  tape  from  // 30 1740- 
25.  Advise  board  replacement. 


Note:  Indicate  TB3  is  replaced. 
On  student  console,  press  FAULT 
PETECT/RETURN. _ 

Rerun  test  tape  from  0301740. 


27.  Student  should  offer  alternative 
explanations  of  malfunction. 
(Instructor  should  only  state  that 
"Yaw  Board  TB3  was  apparently  not 
the  cause.") 


28.  After  5  minutes:  Instructor  suggests 
that  "test  station  caused  the  mal¬ 
function." 

29.  After  another  5  minutes:  Instructor 
suggests  that  "the  test  program 
should  be  decoded." 

30.  Student  decodes  program  in  total. 

31.  Student  decodes  program  in  part. 

32.  Student  identifies  missing  power 
input. 

33.  Student  notes  that  power  light  is 
off. 


34.  Student  suggests  alternative  expla¬ 
nations  for  lack  of  power. 

35.  Students  suggests  that  a  fuse  has 
blown. 


Form  No:  TT01/DRI 
1/79 
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To  test  hypothesis  #2,  the  accuracy  of  groups  A  and  C  will  be 
compared  with  that  of  B  and  D  as  shown  in  Figure  1  on  the  Trouble¬ 
shooting  Test.  In  this  case,  a  2  x  2  analysis-of-variance  will  be  re¬ 
quired  and  a  significant  main  effect  of  training  mode  will  be  necessary 
to  reject  the  hypothesis  that  simulator-trained  and  actual  equipment- 
trained  students  are  equally  accurate  in  solving  trouble-shooting 
prob lems . 

Hypothesis  #3  will  be  tested  by  comparing  the  number  and  nature 
of  errors  committed  and  time  of  completion  for  groups  A  and  C,  and  B 
and  D. 


Hypothesis  #4  will  be  tested  by  comparing  the  accuracy,  time, 
and  error  patterns  of  group  C  with  group  D.  Recall  that  both  of  these 
groups  will  be  tested  only  on  the  actual  equipment.  A  planned  compari¬ 
son  between  groups  C  and  D  is  proposed  to  assess  this  hypothesis.  Of 
course,  the  analysis-of-variance  must  show  some  significant  differences 
in  order  for  the  data  to  potentially  reject  this  hypothesis. 

To  test  hypothesis  #5,  a  modification  will  be  introduced  into 
the  testing  procedure.  Teams  of  students  will  be  created — one  member 
from  each  training  mode.  The  confidence  of  approaching  the  task  will 
be  assessed  by  subjective  ratings  of  each  team  member's  role  in  solving 
the  problem.  This  will  require  some  changes  in  assigning  classes  to 
training  modes.  Therefore,  this  hypothesis  will  be  tested  only  after 
sufficient  data  have  been  collected  to  adequately  assess  all  previous 
hypotheses . 
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COMPONENT  2:  ASSESSMENT  OF  JOB  PROFICIENCY 


As  proposed,  the  major  objective  of  this  aspect  of  the  evalua¬ 
tion  is  to  determine  if  3-level  avionics  maintenance  personnel  perform 
differently  in  the  field  as  a  function  of  the  mode  of  training.  To 
address  this  issue,  three  hypotheses  will  be  tested: 

Hy  6 — Airmen  trained  on  the  6883  simulator  and  the  6883 
test  station  will  acquire  equivalent  job-related 
experience . 

Hy  7 — Airmen  trained  on  the  6883  simulator  and  6883  test 
station  will  be  equally  capable  of  operating  the 
6883  test  station  in  the  field. 

Hy  8 — Airmen  trained  on  the  6883  simulator  and  the  6883 
test  station  are  equally  capable  of  operating 
assigned  test  stations,  other  than  the  6883  station, 
in  the  field. 


Relevant  Issues 


The  performance  of  simulator-trained  personnel  would  be  compared 
to  the  performance  of  actual  equipment-trained  personnel  at  specified 
intervals  of  time  in  the  field.  The  effects  of  on-the-job  training  were 
expected  to  be  present  in  both  groups  and,  therefore,  any  differences  in 
performance  could  be  attributed  to  the  mode  of  training.  Further,  the 
proposed  time  series  sampling  framework  (Figure  4)  was  based  on  the 
assumption  that  student  flow  would  be  45  for  a  one-year  period  as  sug¬ 
gested  in  the  Request  for  Proposal. 

From  our  discussions  with  ATC  staff  and  field  personnel,  a  num¬ 
ber  of  facts  have  emerged  which  must  be  considered  if  we  are  to  appro¬ 
priately  assess  the  impact  of  simulator  training  of  job  performance  in 
the  field.  First,  ATC  now  estimates  the  student  flow  through  the  F-lll 
avionics  maintenance  courses,  which  include  instruction  in  the  operation 
and  maintenance  of  the  6883  test  station,  to  be  approximately  180  airmen 
during  the  1979  calendar  year.  It  has  become  apparent,  however,  that 
field  assignments  are  made  in  view  of  the  momentary  demand  for  specific 
automatic  test  station  operators.  Thus,  at  best,  only  a  small,  undeter¬ 
mined  number  of  airmen  trained  to  operate  the  6883  test  station  can  be 
expected  to  actually  be  assigned  to  the  6883  or  6873  test  stations  in 
the  field.  Further  complicating  the  evaluation  plan  is  the  fact  that 
student  flow  is  not  expected  to  be  at  a  constant  level  throughout  the 
year.  Thus,  the  testing  plan  must  remain  flexible  enough  to  collect 
data  whenever  students  are  available.  The  hypotheses  stated  above 
will  permit  taking  advantage  of  all  simulator-trained  personnel  and 
to  address  the  impact  of  training  on  the  subsequent  operation  of  the 
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Figure  4.  Expected  Sampling  Framework  for  Simulator-Trained  Students 
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Figure  4.  Expected  Sampling  Framework  for  Simulator-Trained  Students 


6883  (or  6873)  test  station  itself,  as  well  as  address  the  issue  of 
generalizability  of  simulator  training  to  the  operation  of  other 
automatic  test  stations.  Second,  field  personnel  have  pointed  out  that 
on-the-job  training  is  tailored  to  the  individual  needs  of  airmen.  The 
amount  of  training  provided  and  the  response  to  that  training  are  not 
formally  documented.  Thus,  we  cannot  expect  the  nature  of  impact  oJ 
on-the-job  training  to  be  consistent  across  groups  much  less  among 
individuals.  On-the-job  training  is  given  to  increase  job  proficiency. 
Promotion,  presumably  based  on  proficiency,  is  therefore  determined  in 
part  by  the  subjective  judgements  of  field  supervisors.  In  short, 
there  is  no  systematic  testing  protocol  currently  implemented  in  the 
field  which  could  be  used  to  track  changes  in  job  proficiency  over  time. 
Our  data  collection  effort  must  be  designed  in  view  of  the  "uncon¬ 
trolled"  nature  of  the  field  environment. 

The  third  issue  affecting  the  data  collection  efforts  in  the 
field  is  that  the  reorientation  of  training  objectives  (discussed  in 
a  previous  section)  requires  a  change  in  the  job  requirements  of  auto¬ 
matic  test  station  operators.  Perhaps  most  important  is  the  fact  that 
the  F-lll  avionics  maintenance  personnel  are  now  expected  to  be  both 
test  station  operators  and  maintainers.  To  be  efficient,  airmen  must 
be  knowledgeable  about  the  relationship  between  LRU  and  test  station 
circuitry.  Since  the  job  requirements  are  somewhat  different  than 
training  objectives,  at  least  in  practice,  it  is  critical  that  a  per¬ 
formance  test  be  developed  in  view  of  field  activities. 


Methodology 

The  groups  shown  in  Figure  1  will  be  considered  as  four  treat¬ 
ment  groups  for  the  purpose  of  assessing  job-related  knowledge.  Since 
this  test  will  be  field-oriented,  and  will  not  directly  mirror  training 
objectives,  it  will  not  be  necessary  to  control  for  the  sequence  of 
training  in  the  6883  practical  block  of  instruction.  At  the  completion 
of  this  block  of  instruction,  the  paper  and  pencil  test  will  be  admin¬ 
istered  to  all  students. 

The  approach  to  collecting  data  in  the  field  will  be  to  compare 
the  subjective  evaluations  of  students  by  field  supervisors  at  three, 
six  and  nine  months  after  field  assignment.  The  eight  groups  for  which 
data  will  be  collected  are  shown  in  Figure  2.  It  will  be  necessary 
to  employ  the  time  series  sampling  framework  for  data  collection,  since 
ATC  graduates  will  be  entering  the  field  at  various  intervals  through¬ 
out  the  year.  A  schedule  of  evaluation  dates  will  be  provided  to 
field  supervisors  for  all  airmen  in  the  original  four  treatment  groups. 


Data  Collection  Instruments 


Two  data  collection  instruments  will  be  required  to  test  the 
three  hypotheses  associated  with  this  component  of  the  evaluation 
plan:  the  Estimated  Job  Proficiency  Test  and  the  Field  Performance 
Inventory . 

The  Estimated  Job  Proficiency  Test  consists  of  items  created 
by  field  supervisors  and  other  selected  field  personnel.  The  items 
are  designed  to  determine  the  level  of  knowledge  and  skills  deemed 
necessary  for  adequate  performance  in  the  field.  So  constructed,  the 
test  is  expected  to  address  performance  ability  beyond  training  objec¬ 
tives  or  Specialty  Training  Standards.  This  test  will  be  administered 
to  all  students  upon  completion  of  the  6883  practical  block  of 
instruction. 

The  Field  Performance  Inventory  will  be  designed  as  a  question¬ 
naire  to  be  completed  by  the  field  supervisor.  The  items  will  address 
field  performance  of  graduates  of  course  3ARB32634A  after  three,  six, 
and  nine  months  in  the  field.  The  questionnaire  will  be  designed  so 
that  consistent  data  will  be  collected  at  each  time  interval. 


Hypothesis  Testing 

To  test  hypothesis  #6,  test  scores  on  the  Estimated  Job 
Proficiency  Test  will  be  compared  for  the  four  basic  groups  shown  in 
Figure  1.  The  hypothesis  will  be  rejected  if  a  significant  main  effect 
of  training  mode  is  obtained. 

To  test  hypothesis  #7,  only  airmen  assigned  to  the  6883  or  6873 
test  station  will  be  considered.  For  this  group,  a  comparison  of  sub¬ 
jective  performance  ratings  will  be  made  for  students  trained  on  the 
6883  simulator  versus  those  trained  on  the  actual  equipment. 

The  test  of  hypothesis  #8  will  be  identical  to  that  for 
hypothesis  #7,  except  that  only  airmen  assigned  to  automatic  test  sta¬ 
tions  other  than  the  6883  or  6873  will  be  considered.  Again,  subjec¬ 
tive  performance  ratings  will  be  compared  to  isolate  any  differences 
due  to  mode  of  training. 
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COMPONENT  3:  COST  ANALYSIS 


'  1,1,1  ‘ 

~ - 


As  originally  proposed,  the  purpose  of  this  component  of  the 
evaluation  effort  is  to  document  all  significant  cost  associated  with 
the  use  of  the  6883  simulator  and  the  actual  test  station  equipment  for 
training.  The  resulting  comparative  cost  analysis  will  provide  data 
which  can  be  used  for  making  decisions  concerning  the  procurement  and 
use  of  similar  equipment  in  the  future. 


Methodology 

In  order  to  conduct  a  systematic  comparison  of  costs,  a  basic 
model  or  framework  has  been  developed  which  includes  the  six  major 
cost  categories  shown  in  Table  5 .  The  basic  model  is  a  matrix  which 
has  cost  categories  and  life  cycle  as  the  two  primary  dimensions.  To 
conduct  an  analysis,  a  detailed  matrix  which  includes  specific  line 
items  for  each  major  category  will  be  completed.  These  detailed  cost 
breakdowns  for  each  of  the  six  major  categories  are  shown  in  Tables 
6-11. 


TABLE  5 .  BASIC  COST  COMPARISON  MODEL 


Cost  Categories 

Life  Cycle 

R&D 

Investment 

Purchase/Supplemental  Start-up 

Operating 
Years  1-n 

Facilities 

Equipment 

Instructional 
Material /Training 

Instructors 

Students 

Miscellaneous 

It  should  be  noted  that  the  line  items  selected  will  allow 
the  model  to  be  consistently  applied  in  a  wide  variety  of  situations 
where  a  choice  between  the  use  of  simulated  and  actual  test  station 
equipment  for  training  might  be  made.  It  is  not  unlikely,  therefore, 
that  in  any  given  application  of  the  model,  many  of  the  line  items  will 
be  considered  irrelevant  or  sunk  costs.  In  these  cases,  they  would 
be  recorded  as  zero  costs.  For  example,  one  of  the  life  cycle  cate¬ 
gories  is  research  and  development  (R&D) .  To  estimate  the  cost  of 
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TABLE  6.  LIKE  ITEMS  ASSOCIATED  WITH  FACILITIES  CATECORT 


12.  Ocher 


TABLE  8.  LINE  ITEMS  ASSOCIATED  WITH  INSTRUCTIONAL  MATERIALS /TRAINING  CATECORY 
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TABLE  10.  LINE  ITEMS  ASSOCIATED  WITH  STUDENTS  CATECORY 


TABLE  11.  line  ITEMS  ASSOCIATED  WITH  MISCELLANEOUS  CATEGORY 


acquiring  a  simulator,  R&D  incurred  by  the  manufacturer  may  not  be 
separately  identifiable  to  the  purchaser  and  may  be  simply  included  in 
the  purchase  price.  R&D  costs  associated  with  an  actual  test  station 
were  most  likely  incurred  as  part  of  the  cost  of  instrument  development 
for  field  use.  Thus,  in  estimating  the  cost  of  acquiring  an  actual 
test  station  for  training,  it  would  be  inappropriate  to  include  R&D 
costs.  An  example  of  sunk  cost  would  be  the  previous  acquisition  cost 
of  actual  test  station  equipment  which  is  involved  in  the  decision  of 
whether  to  continue  operation  of  that  equipment  or  to  purchase  and 
operate  a  simulated  test  station.  In  short,  the  determination  of  which 
costs  are  relevant  is  completely  dependent  on  the  decision  being  made 
at  a  specific  point  in  time. 


Application  of  the  Model 

A  model  is  an  intermediate  step  between  inputs  and  outputs;  a 
model  by  itself  cannot  generate  the  necessary  cost  data.  The  inputs 
and  their  values  have  to  be  externally  derived.  In  some  cost  cate¬ 
gories,  the  value  of  an  input  can  be  accurately  determined,  while  others 
can  only  be  estimated.  That  is,  actual  costs  could  be  obtained  from 
competitive  bids  from  qualified  contractors  whereas  cost  estimates  can 
be  made  on  the  basis  of  previous  experience  or  historical  records. 

Realizing  that  the  output  of  the  cost  model  is  only  as 
reliable  as  the  data  input,  the  following  five  steps  in  applying  the 
model  in  s  given  context  are  essential: 

1.  Define  the  question  to  be  answered.  This  step  is  crucial 
because  the  decision  to  be  made  will  determine  which  cost  categories 
and  line  items  are  relevant. 

2.  Establish  life  cycle.  Fifteen  years  is  the  recommended 
life  cycle  since  that  is  consistent  with  past  Air  Force  policy. 

3.  Determine  or  estimate  input  values.  This  step  may  be  the 
most  time  consuming.  Some  line  items  may  only  be  estimated  as  a  per¬ 
centage  of  a  reference  cost  (e.g.,  parts  as  a  percentage  of  purchase 
price).  In  addition,  there  may  be  some  concern  regarding  purchase 
options  not  essential  to  meet  stated  training  objectives.  An  actual 
test  station  may  be  available  which  is  significantly  overdesigned  for 
training  purposes.  In  assessing  the  simulated  equipment  alternative, 
a  similarly  overdesigned  test  station  or  a  simulator  designed  in  view 
of  course  objectives  might  be  considered.  To  illustrate,  consider 
that  a  basic  simulator  capability  is  available  at  a  base  price  of 
$300,000.  Ten  additional  capabilities  will  cost  $50,000  each.  However, 
only  four  capabilities  are  needed  to  provide  training  consistent  with 
course  objectives.  The  total  purchase  price  of  this  alternative  is 
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$300,000  +  (4) ($50 ,000)  =  $500,000.  On  the  other  hand,  the  actual  test 
station  available  costs  $1  million;  but  50  percent  of  the  price  repre¬ 
sents  capabilities  beyond  those  needed  for  the  course.  Nevertheless, 
the  relevant  price  of  this  alternative  is  $1  million  unless  unwanted 
capabilities  can  be  used  to  meet  other  training  requirements,  in  which 
case,  appropriate  cost  adjustments  can  be  made. 

4.  Exercise  the  model.  All  actual  or  estimated  costs  are 
input  into  the  model  and  all  irrelevant  costs  are  considered  zero. 

5.  Express  result  in  present  value,  using  accepted  discount 
factor.  Different  cost  streams  may  have  different  impacts,  depending 
upon  the  time  at  which  costs  are  incurred.  Present  value  is  a  widely 
accepted  method  of  accounting  for  differences  in  time  patterns. 


SUMMARY  OF  HYPOTHESES  TO  BE  TESTED 


Table  12  shows  all  hypotheses  to  be  tested  and  the  sample, 
data  collecting  instrument,  and  measure  to  be  used.  The  time  of  data 
collection  is  also  indicated. 

It  should  be  noted  that  additional  hypotheses  could  be  consi¬ 
dered  as  the  evaluation  progresses.  This  could  result  from  changes  in 
ATC  activities  or  the  emergence  of  an  apparent  unanticipated  impact 
which  must  be  verified.  The  assessment  of  additional  hypotheses 
should  involve  no  new  data  collection  instruments  or  increased  demands 
on  ATC  or  field  personnel.  Rather,  existing  procedures  will  be  modi¬ 
fied  to  accommodate  any  new  evaluation  requirements. 
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TABLE  12.  SUMMARY  OF  HYPOTHESES  AND  RELATED  DATA  COLLECTION  EFFORTS 
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6883  test  station  are  test  stations  other  performance  the  field 

equally  capable  of  oper-  than  the  6883  or 
ating  assigned  test  sta-  6873 
tions,  other  than  the 

6883  station,  in  the  field. _ 
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lator,  identify  the  function(s) 
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Given  a  block  diagram  of  the  27b9.  Theory  and  signal  flow  of  the  2a 

Digital  Interface  Unit  and  the  digital  interface  unit 

appropriate  TO,  identify  the 
function  of  the  blocks. 


03 

03 

03 

03 

CD 

03 

X 

X 

x 

-C 

•e 

X 

4-1 

4-1 

u 

4-3 

4— > 

4-1 

4-4 

4-4 

4-4 

4-4 

4-1 

4-4 

O 

O 

o 

o 

O 

O 

3  hj 

3  hi 

3  4-1 

3 

3 

3 

O  *H 

O  -H 

0  -H 

o 

o 

o 

rH  3 

t-H  P 

*-H  P 

t-H 

t-H 

t-H 

4-4  P 

U-l  P 

4-4  p 

4-4 

4-4 

4-4 

03 

o  3 

t-H  t-H 

t-H  rH 

t-H  rH 

t-H 

rH 

rH 

3  O 

03  O 

03  O 

03  O 

03 

03 

CU 

03  -H 

C  u 

c  U 

e  n 

P 

P 

P 

M  4J 

00  4-J 

00  4-J 

DO  4-3 

00 

00 

co 

cd  a 

•H  C 

•H  P 

•H  P 

•H 

•H 

•H 

4-4  -H 

73  O 

73  O 

73  O 

73 

73 

73 

03  U 

U 

O 

CD 

V-i 

V4 

os  a 

03 

X 

X 

03  03 

03  03 

X  03 

73 

c  oo 

P  00 

p  oo 

3  HI 

3  4-4 

P  4-J 

<  03 

03  3 

03  P 

03  3 

o3  a- 

03  r- 

03  a 

<r  Q 

•H 

•H 

•H 

03 

03 

03 

X 

X  4= 

X 

X 

03 

X 

03 

1  OD 

3-<  U 

S-4  U 

J-4  U 

b  CO 

U  03 

3  03 

co  a 

O  4.3 

O  -i-l 

O  4-1 

0 

o 

O 

f-n  03 

03  -H 

03  *H 

03  *H 

03  CO 

03  CO 

03  CO 

co 

x:  3 

X  3 

X  3 

£  o 

x:  o 

X  o 

H  c/3 

H  C/3 

H  co 

H  Eh 

H  b. 

H  b. 

CM 

CM 

CM 

'd' 

<r 

<r 

£3 

£3 

£3 

£3 

£3 

£3 

r- 

r". 

CM 

CM 

CM 

CM 

CM 

CM 

CD 

X 

4-J  Q3 

X 

M  4-> 

o  x 

in  TJ  U 
3  03 
C/3  03  CJ 


CO  C  O 
O  3D  U 
U  03 
G.H  TJ 
O 

4-4  P  * 

O  hj  O 


4-1  CD  00 

O  4-1  C 
03  *H 
3  *H  x: 
o  u  u 

•H  CL  4J  . 


CD  CD 

x  *  x 

4-3  O  4-1 

H 


nj  *h  4  E 
£  a  oj 

3  0  0  3-4 
d)  n  D  OC 

>  -h  a  o 
•h  ^  a  m 
o  c/3  03  a 


3  o 

C  -H  4-( 


•H  J3  O 

a  4J  h 


CL  £  £ 

c  o  u  s 

CD  Lw  4-» 

>  cl  -h  3 

•H  a,  3  *H 

o  ni  w  a 


03  *H  73 
3  4-1  4-J  ^ 

oj  a.  3  u 

>  03  03  O 
•H  T3  XJ  H 
O  <  *H  .O 


•H 

Vn  3 

X 

03 

a 

73 

4-1 

4-4 

4-4 

o  a 

73 

LJ 

• 

T3 

CL  CO 

CD 

4-3 

73  4-J 

•H 

O 

O 

03  o 

H  Ch 

g 

CL 

B 

3 

CL 

P 

rH  03 

4-J 

4-J 

3 

O- 

3 

o 

03  03 

•— 

4-1 

03  O 

•H 

rH 

73 

B 

03  P 

03  03 

Hi 

03 

U 

o 

x 

s 

o 

C  *H 

3 

03 

•H 

03 

•H  o 

4J  X 

00 

00 

03  -U 

H 

00  03 

33 

3 

73 

Hi 

Hi  *H 

03  4-J 

o 

03 

O 

• 

X 

03 

4-3 

•H  C 

00 

th 

OO 

a  4-» 

•H 

*H 

X 

u 

u 

■U  4-( 

4-1 

3 

73  *H 

rH 

•H 

rH 

03 

o  a 

Hi  X 

CL 

4-J 

CL 

0 

03 

a 

o 

73 

03 

•H 

M  3 

a-  oo 

4-J 

03 

03 

4-3 

4-4  r> 

Hi 

P 

03 

CL  P 

O  3 

4-4 

03 

X 

03 

p  Pi 

3 

0  o 

u 

X 

03 

CL  4-4 

u  0 

o 

3 

u 

£ 

03  0d 

* 

O 

H 

3 

a 

X 

3 

a  m 

03 

3 

03 

CO 

4-J 

4J 

O 

03 

2 

a 

Q) 

a  x 

4-3 

03 

X 

n 

•H 

03 

73  CD 

CJ 

03 

O 

0 

03  X 

0)  4-J 

73 

Lj 

u 

C/3  P 

P 

x 

•H  4J 

rH 

t-H 

3  4J 

•H 

03 

a i 

73 

4J  03 

4-J 

rH  03 

00 

CD 

4-4 

X 

3 

cd  3 

rH 

4-» 

03 

3 

p 

u 

>> 

X  0 

a 

0 

03 

a  u 

rH 

4-4 

CQ  Hi 

•H 

03 

03 

03 

Hi  4-< 

4-3  rH 

3 

3 

a 

03  03 
3  <  03 


•H  CJ  O 
O  tn  H 


Current  STS  Prior  Status 


H5 

O  /— s 
— t  < 

cq  <r 
X 
—i  \ 
CT3  a 
O  rH 
•H  X 
4-J 

u  o> 

03  co 


O  4-1 
U 

co  P 

03  J-4 


E  ° 


03 

^  cm 


I 

a 


u 

co 

sO  cm  »— < 


<r 

X 
X  r- 
vD  CM 


a 

a 

O 

4-J 

1 

o 

1 

X 

o 

o 

CO 

CO 

CO 

O 

c 

pC 

CO 

03 

CO 

CO 

03 

CO 

1 

CO 

u 

03 

4-1  • 

03 

•H 

00 

o 

E 

rH 

X 

03 

X 

O 

X 

O  -H 

X 

o 

03 

e 

•H 

c 

X 

C 

4-J  U 

c 

4-J 

03 

03 

ts 

0. 

CO 

03 

4-> 

03 

Uh 

1 

4-J 

4J 

C 

e 

c 

03 

c 

C  03 

C 

c 

Uh 

o 

c 

o 

rH 

o 

03  C 

o 

03 

0 

cu 

•H 

o 

•H 

X 

•H 

e  « 

•H 

r 

u 

c 

4-J 

Uh 

4-J 

P 

4-J 

a 

4-J 

a 

CO 

c 

o 

03 

3h 

03 

o 

03 

•H  r— 1 

03 

•H 

c 

o 

•H 

4-1 

03 

• 

4-J 

u 

4-J 

P  03 

4-1 

P 

o 

u 

u 

CO 

a 

00 

CO 

4-J 

CO 

cr  03 

CO 

cr 

•H 

03 

a 

c 

• 

03  Cm 

03 

4-» 

a 

•H 

4-J 

o 

•H 

■u 

o 

CO 

4-J 

4-J 

u 

S3 

u 

CO 

4-1 

4-» 

CO 

4-J 

c 

CO 

4-J  4-J 

CO 

4-J 

C 

a 

u 

03 

CO 

03 

o 

03 

CO  u 

03 

CO 

P 

03 

4-1 

CO 

03 

4-1 

CO 

•H 

4-J 

0)  03 

4-J 

03 

U-4 

< 

0) 

X 

4-J 

X 

4-J 

4-»  a 

4-J 

rH 

« 

0) 

03 

>H 

CJ 

03 

CO 

03 

03 

X 

4-1 

03 

o 

4-J 

03 

C 

4-J 

X  G 

4-J 

X 

E 

1 

X 

03 

03 

•H 

03 

X 

P 

03 

03  *H 

03 

03 

• 

m 

C 

>H 

C 

4-1 

U 

C 

CM 

u 

4-J 

U 

4-J 

03 

B 

H 

03 

0) 

03 

CO 

03 

03 

r—4 

03 

03  X 

03 

03 

4-J 

•H 

cn 

a 

4-J 

O 

a 

4-J 

03 

a 

•H  C 

a 

•H 

03 

o 

CO 

c 

O 

CO 

e 

o 

U  03 

o 

U 

4-J 

pH 

CM 

i— H 

r—i 

x 

X 

u 

03 

r-* 

00 

00 

_ 

_ 

CM 

_ 

CM 

_ 

CM 

CM 

CM  1 

03 

CO 

•H 

03 

X 

4-J 

X 

• 

P 

03 

B 

a 

c 

U 

03 

C/3 

03 

o 

M 

00 

cr 

pC 

o 

03 

o 

4-1 

''■V. 

03 

in 

03 

C 

03 

4-J 

X 

• 

0» 

CO 

* 

CO 

0! 

X 

Hi 

*J 

X 

•H 

a 

4-J 

03 

CO 

4-J 

03 

CO 

rH 

G 

pC 

c 

C 

4-J 

4-J 

03 

4— ! 

f-H 

4-J 

03 

CO 

• 

4-J 

03 

O 

03 

• 

o 

o 

a 

CO 

CO 

4-J 

a 

a 

03 

03 

E 

03 

CO 

4-J 

03 

o 

4-J 

03 

03 

03 

0) 

X 

X 

4-J 

•H 

X 

X 

•H 

rH 

c 

4-J 

CO 

4-J 

4-J 

U 

i — ‘ 

• 

C 

u 

c 

03 

o 

• 

0 

03 

> 

e 

X 

03 

o 

4-J 

pG 

03 

c 

03 

o 

03 

u 

a 

u 

03 

»H 

CO 

c 

• 

•H 

u 

4-J 

O 

4-J 

U-i 

4-J 

i-H 

o 

•H 

4-J 

o 

c 

03 

3-J 

03 

C 

X 

03 

C 

• 

03 

a 

X 

4—1 

03 

r 

3-4 

CJ 

o 

CJ 

H 

4-J 

O 

4-J 

c 

4-J 

o 

4-J 

03 

4-J 

•H 

CO 

CO 

•H 

03 

O 

•H 

G 

•H 

c 

03 

•H 

c 

pG 

• 

o 

4-J 

4-J 

03 

4-J 

03 

X 

03 

4-J 

03 

03 

4-J 

03 

4-J 

a 

a 

> 

X 

c 

a 

03 

u 

.a 

03 

C 

C 

4-< 

03 

B 

rH 

4-J 

03 

E 

oc 

o 

c 

c 

00 

o 

X 

p 

4-J 

4-J 

03 

03 

03 

4-J 

a 

03 

03 

4-J 

a  <4-i 

• 

c 

M 

o 

03 

03 

3-4 

3h 

CO 

4-J 

•H 

CO 

*H 

03 

•H 

CO 

•H 

o 

>1 

•H 

a 

cj 

•H 

a 

X 

4-J 

3-4 

c 

r— 1 

3h 

P 

<M 

U 

P 

c. 

4-J 

a 

c 

X 

a 

u 

CO 

O 

4-J 

•H 

03 

a 

4-J 

cr 

a 

4-J 

cr 

CO 

O1 

CO 

C0 

03 

o 

03 

0j 

c 

<4-4 

CO 

03 

0) 

o 

CO 

0) 

4-J 

o 

CO 

03 

G 

u 

03 

X 

•H 

g 

4J 

•H 

03 

a 

3h 

03 

O 

u 

03 

o 

H 

03 

4-> 

4-1 

Jh 

03 

CO 

H 

a 

4-J 

4-J 

03 

a 

4-J 

4-J 

<M 

X 

03 

o 

X 

X 

<M 

c 

| 

<M 

a 

CO 

a 

a 

CO 

4-J 

O 

03 

4-J 

03 

4-J 

4h 

4-J 

c 

O 

o 

*H 

o 

03 

03 

0) 

CO 

03 

0> 

03 

u 

U 

4— J 

m 

M 

03 

•H 

O 

C 

4-J 

4-J 

G 

4-J 

4-J 

G 

gj 

C 

c 

03 

03 

a 

03 

4-J 

M 

o 

c 

C 

03 

•H 

c 

03 

P 

o 

03 

03 

>H 

4-J 

a 

03 

4-J 

CO 

U 

4-J 

•H 

o 

03 

U 

X 

0) 

3h 

X 

•H 

G 

> 

03 

CO 

> 

c 

P 

C 

c 

4-J 

•H 

> 

03 

t) 

X 

> 

03 

03 

rH 

4-J 

01 

•H 

a 

03 

o 

•H 

03 

OJ 

P 

o 

03 

4-J 

♦H 

a 

4-J 

G 

•H 

a 

4-J 

03 

03 

4-J 

o 

o 

H 

4-1 

o 

E 

u 

<M 

CJ 

P 

03 

o 

o 

03 

03 

o 

o 

03 

B 

a 

C 

f-H 

3h 

03 1 

•H 

03 

03 

aJ 

03 

• 

• 

> 

a 

• 

• 

X 

03 

pO 

a 

o| 

03 

X 

6 

•H 


58 


Evaluation  Feel  and  Trim 


POI  Current  STS  Prior  Status 


APPENDIX  B 

FORMER  6883  INSTRUCTION  BLOCK  OBJECTIVES 
DELETED  FROM  CURRENT  BLOCK 
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FORMER  6883  BLOCK  OBJECTIVES  NOT  INCLUDED  IN  COURSE  X4A 


6883  block  objectives  of  course  X1D  which  address  Current  X4A 

X1D  STS's  which  are  retained  in  STS  X4A  but  are  not  STS  Reference 

addressed  by  POI  for  new  course 

3.  Data  flow  analysis  of  the  flight  control  computers 

a.  Provided  with  TO  5A7-3-14-2  and  a  list  of  selected  28a4 
Yaw  Computer  subassemblies,  state  the  function  of 

each . 

b.  Provided  with  TO  5A7-3-14-2  and  a  list  of  selected  28b4 
Yaw  Computer  signals,  state  the  plug  and  pin  num¬ 
bers  where  the  signals  appear. 

c.  Provided  with  TO  5A7-3-15-2  and  a  TO  5A7-3-16-2  28a4 

and  a  list  of  selected  Pitch  and  Roll  Computer 
subassemblies,  state  the  function  of  each. 

d.  Provided  with  TO  5A7-3-15-2  and  TO  5A7-3-16-2  and  28b4 

a  list  of  selected  Pitch  and  Roll  Computer  sig¬ 
nals,  state  the  plug  and  pin  numbers  where  the 

signals  appear. 

5.  Theory  of  operation  of  the  converter-flight  controls 

system  test  station 

*a.  Provided  with  TO  33D7-17-15-2  and  a  list  of  16 

selected  test  station  operation  procedures, 
state  the  methods  of  operation. 

6. 

c.  Provided  with  TO  5A9-2-42-22  and  TO  5A9-2  42-28-1  28el 

and  the  previously  identified  causes  of  known 
feel  and  trim  assembly  malfunctions,  perform  the 
repair  action. 

f.  Provided  with  TO  11F15-3-17-2  and  TO  1 1 F 15-3-1 7-8- ]  28e3 

and  the  previously  identified  causes  of  known 
multiplexer  converter  malfunctions,  perform  the 
repair  action. 


*This  objective  may  be  judged  to  be  included  since  16  is  referenced 
in  the  POI  for  course  X4A. 


6883  block  objectives  of  course  X1D  which  address 

X1D  STS's  which  are  retained  in  STS  X4A  but  are  not 
addressed  by  POI  for  new  course 

Current  X4A 
STS  Reference 

6.  g.  Provided  with  TO  5A7-3-14-8-1 ,  state  the  proce- 

28c4 

dures  to  inspect,  connect,  and  test  the  Yaw 

flight  control  computer. 

h.  Provided  with  TO  5A7-3-14-8-1  and  selected  mal- 

28d4 

functions  of  the  Yaw  flight  control  computer. 

state  the  cause  of  each  malfunction. 

6883  block  objectives  of  course  XOB  which  address 

XOB  STS's  which  are  retained  in  STS  X4A  but  are  not 

Current  X4A 

addressed  by  POI  for  new  course 

STS  Reference 

NOTE:  These  objectives  are  in  practical  block  of 

instruction. 

2.  Calibration/verification,  alignment  and  trouble- 

shooting  analysis  of  the  converter-flight  control 

systems  test  station. 

a.  Using  TO  33D7-17-15-2,  perform  verif ication/cali- 

27d4 

bration  of  peculiar  TRU's  of  the  converter-flight 

control  systems  test  station. 

**c.  With  instructor  assistance,  isolate  the  cause  of 

27d3 

two  specified  NO-GO  conditions  following  proce- 

dures  listed  in  TO  33D7-17-15-8-1  to  the  lowest 

replaceable  component. 

6883  block  objectives  of  course  X1D  INTERIM  which 

address  STS  requirements  that  are  related  in  STS  X4A 

Current  X4A 

but  are  not  addressed  by  POI  for  new  course 

STS  Reference 

6f.  See  objectives  for  XlD. 

28e3 

1.  (practical) 

Given  TO  33D7-47-13-2  and  TO  33D7-47-13-8-1 ,  perform 

27c 

Turn-on  procedures  and  confidence  testing. 

**It  is  not  clear  that  this  objective  was  ever  addressed  in  refer¬ 
enced  task.  The  reference  also  shows  up  in  objective  la. 
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